GitHub |
您所在的位置：网站首页 › point cloud neural network › GitHub |
GitHub
|
Documentation | Blog | Demo Learning3D is an open-source library that supports the development of deep learning algorithms that deal with 3D data. The Learning3D exposes a set of state of art deep neural networks in python. A modular code has been provided for further development. We welcome contributions from the open-source community. Latest News: [12 May, 2022]: ChamferDistance loss function is incorporated in learning3d. This is a purely pytorch based loss function. [24 Dec. 2020]: MaskNet is now ready to enhance the performance of registration algorithms in learning3d for occluded point clouds. [24 Dec. 2020]: Loss based on the predicted and ground truth correspondences is added in learning3d after consideration of Correspondence Matrices are Underrated paper. [24 Dec. 2020]: PointConv, latent feature estimation using convolutions on point clouds is now available in learning3d. [16 Oct. 2020]: DeepGMR, registration using gaussian mixture models is now available in learning3d [14 Oct. 2020]: Now, use your own data in learning3d. (Check out UserData functionality!) Available Computer Vision Algorithms in Learning3D Sr. No. Tasks Algorithms 1 Classification PointNet, DGCNN, PPFNet, PointConv 2 Segmentation PointNet, DGCNN 3 Reconstruction Point Completion Network (PCN) 4 Registration PointNetLK, PCRNet, DCP, PRNet, RPM-Net, DeepGMR 5 Flow Estimation FlowNet3D 6 Inlier Estimation MaskNet Available Pretrained Models PointNet PCN PointNetLK PCRNet DCP PRNet FlowNet3D RPM-Net (clean-trained.pth, noisy-trained.pth, partial-pretrained.pth) DeepGMR PointConv (Download from this link) MaskNet Available Datasets ModelNet40 Available Loss Functions Classification Loss (Cross Entropy) Registration Losses (FrobeniusNormLoss, RMSEFeaturesLoss) Distance Losses (Chamfer Distance, Earth Mover's Distance) Correspondence Loss (based on this paper) Technical Details Supported OS Ubuntu 16.04 Ubuntu 18.04 Ubuntu 20.04.6 Linux Mint Requirements CUDA 10.0 or higher Pytorch 1.3 or higher Python 3.8 How to use this library?Important Note: Clone this repository in your project. Please don't add your codes in "learning3d" folder. All networks are defined in the module "models". All loss functions are defined in the module "losses". Data loaders are pre-defined in data_utils/dataloaders.py file. All pretrained models are provided in learning3d/pretrained folder. DocumentationB: Batch Size, N: No. of points and C: Channels. Use of Point Embedding Networks:from learning3d.models import PointNet, DGCNN, PPFNet pn = PointNet(emb_dims=1024, input_shape='bnc', use_bn=False) dgcnn = DGCNN(emb_dims=1024, input_shape='bnc') ppf = PPFNet(features=['ppf', 'dxyz', 'xyz'], emb_dims=96, radius='0.3', num_neighbours=64) Sr. No. Variable Data type Shape Choices Use 1. emb_dims Integer Scalar 1024, 512 Size of feature vector for the each point 2. input_shape String - 'bnc', 'bcn' Shape of input point cloud 3. output tensor BxCxN - High dimensional embeddings for each point 4. features List of Strings - ['ppf', 'dxyz', 'xyz'] Use of various features 5. radius Float Scalar 0.3 Radius of cluster for computing local features 6. num_neighbours Integer Scalar 64 Maximum number of points to consider per cluster Use of Classification / Segmentation Network:from learning3d.models import Classifier, PointNet, Segmentation classifier = Classifier(feature_model=PointNet(), num_classes=40) seg = Segmentation(feature_model=PointNet(), num_classes=40) Sr. No. Variable Data type Shape Choices Use 1. feature_model Object - PointNet / DGCNN Point cloud embedding network 2. num_classes Integer Scalar 10, 40 Number of object categories to be classified 3. output tensor Classification: Bx40, Segmentation: BxNx40 10, 40 Probabilities of each category or each point Use of Registration Networks:from learning3d.models import PointNet, PointNetLK, DCP, iPCRNet, PRNet, PPFNet, RPMNet pnlk = PointNetLK(feature_model=PointNet(), delta=1e-02, xtol=1e-07, p0_zero_mean=True, p1_zero_mean=True, pooling='max') dcp = DCP(feature_model=PointNet(), pointer_='transformer', head='svd') pcrnet = iPCRNet(feature_moodel=PointNet(), pooling='max') rpmnet = RPMNet(feature_model=PPFNet()) deepgmr = DeepGMR(use_rri=True, feature_model=PointNet(), nearest_neighbors=20) Sr. No. Variable Data type Choices Use Algorithm 1. feature_model Object PointNet / DGCNN Point cloud embedding network PointNetLK 2. delta Float Scalar Parameter to calculate approximate jacobian PointNetLK 3. xtol Float Scalar Check tolerance to stop iterations PointNetLK 4. p0_zero_mean Boolean True/False Subtract mean from template point cloud PointNetLK 5. p1_zero_mean Boolean True/False Subtract mean from source point cloud PointNetLK 6. pooling String 'max' / 'avg' Type of pooling used to get global feature vectror PointNetLK 7. pointer_ String 'transformer' / 'identity' Choice for Transformer/Attention network DCP 8. head String 'svd' / 'mlp' Choice of module to estimate registration params DCP 9. use_rri Boolean True/False Use nearest neighbors to estimate point cloud features. DeepGMR 10. nearest_neighbores Integer 20/any integer Give number of nearest neighbors used to estimate features DeepGMR Use of Inlier Estimation Network (MaskNet):from learning3d.models import MaskNet, PointNet masknet = MaskNet(feature_model=PointNet(), is_training=True) Sr. No. Variable Data type Choices Use 1. feature_model Object PointNet / DGCNN Point cloud embedding network 2. is_training Boolean True / False Specify if the network will undergo training or testing Use of Point Completion Network:from learning3d.models import PCN pcn = PCN(emb_dims=1024, input_shape='bnc', num_coarse=1024, grid_size=4, detailed_output=True) Sr. No. Variable Data type Choices Use 1. emb_dims Integer 1024, 512 Size of feature vector for each point 2. input_shape String 'bnc' / 'bcn' Shape of input point cloud 3. num_coarse Integer 1024 Shape of output point cloud 4. grid_size Integer 4, 8, 16 Size of grid used to produce detailed output 5. detailed_output Boolean True / False Choice for additional module to create detailed output point cloud Use of PointConv:Use the following to create pretrained model provided by authors. from learning3d.models import create_pointconv PointConv = create_pointconv(classifier=True, pretrained='path of checkpoint') ptconv = PointConv(emb_dims=1024, input_shape='bnc', input_channel_dim=6, classifier=True) OR Use the following to create your own PointConv model. PointConv = create_pointconv(classifier=False, pretrained=None) ptconv = PointConv(emb_dims=1024, input_shape='bnc', input_channel_dim=3, classifier=True) PointConv variable is a class. Users can use it to create a sub-class to override create_classifier and create_structure methods in order to change PointConv's network architecture. Sr. No. Variable Data type Choices Use 1. emb_dims Integer 1024, 512 Size of feature vector for each point 2. input_shape String 'bnc' / 'bcn' Shape of input point cloud 3. input_channel_dim Integer 3/6 Define if point cloud contains only xyz co-ordinates or normals and colors as well 4. classifier Boolean True / False Choose if you want to use a classifier with PointConv 5. pretrained Boolean String Give path of the pretrained classifier model (only use it for weights given by authors) Use of Flow Estimation Network:from learning3d.models import FlowNet3D flownet = FlowNet3D() Use of Data Loaders:from learning3d.data_utils import ModelNet40Data, ClassificationData, RegistrationData, FlowData modelnet40 = ModelNet40Data(train=True, num_points=1024, download=True) classification_data = ClassificationData(data_class=ModelNet40Data()) registration_data = RegistrationData(algorithm='PointNetLK', data_class=ModelNet40Data(), partial_source=False, partial_template=False, noise=False) flow_data = FlowData() Sr. No. Variable Data type Choices Use 1. train Boolean True / False Split data as train/test set 2. num_points Integer 1024 Number of points in each point cloud 3. download Boolean True / False If data not available then download it 4. data_class Object - Specify which dataset to use 5. algorithm String 'PointNetLK', 'PCRNet', 'DCP', 'iPCRNet' Algorithm used for registration 6. partial_source Boolean True / False Create partial source point cloud 7. partial_template Boolean True / False Create partial template point cloud 8. noise Boolean True / False Add noise in source point cloud Use Your Own Data:from learning3d.data_utils import UserData dataset = UserData(application, data_dict) Sr. No. Application Required Key Respective Value 1. 'classification' 'pcs' Point Clouds (BxNx3) 'labels' Ground Truth Class Labels (BxN) 2. 'registration' 'template' Template Point Clouds (BxNx3) 'source' Source Point Clouds (BxNx3) 'transformation' Ground Truth Transformation (Bx4x4) 3. 'flow_estimation' 'frame1' Point Clouds (BxNx3) 'frame2' Point Clouds (BxNx3) 'flow' Ground Truth Flow Vector (BxNx3) Use of Loss Functions:from learning3d.losses import RMSEFeaturesLoss, FrobeniusNormLoss, ClassificationLoss, EMDLoss, ChamferDistanceLoss, CorrespondenceLoss rmse = RMSEFeaturesLoss() fn_loss = FrobeniusNormLoss() classification_loss = ClassificationLoss() emd = EMDLoss() cd = ChamferDistanceLoss() corr = CorrespondenceLoss() Sr. No. Loss Type Use 1. RMSEFeaturesLoss Used to find root mean square value between two global feature vectors of point clouds 2. FrobeniusNormLoss Used to find frobenius norm between two transfromation matrices 3. ClassificationLoss Used to calculate cross-entropy loss 4. EMDLoss Earth Mover's distance between two given point clouds 5. ChamferDistanceLoss Chamfer's distance between two given point clouds 6. CorrespondenceLoss Computes cross entropy loss using the predicted correspondence and ground truth correspondence for each source point To run codes from examples: Copy the file from "examples" folder outside of the directory "learning3d" Now, run the file. (ex. python test_pointnet.py) Your Directory/Location learning3d test_pointnet.py References: PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation Dynamic Graph CNN for Learning on Point Clouds PPFNet: Global Context Aware Local Features for Robust 3D Point Matching PointConv: Deep Convolutional Networks on 3D Point Clouds PointNetLK: Robust & Efficient Point Cloud Registration using PointNet PCRNet: Point Cloud Registration Network using PointNet Encoding Deep Closest Point: Learning Representations for Point Cloud Registration PRNet: Self-Supervised Learning for Partial-to-Partial Registration FlowNet3D: Learning Scene Flow in 3D Point Clouds PCN: Point Completion Network RPM-Net: Robust Point Matching using Learned Features 3D ShapeNets: A Deep Representation for Volumetric Shapes DeepGMR: Learning Latent Gaussian Mixture Models for Registration CMU: Correspondence Matrices are Underrated MaskNet: A Fully-Convolutional Network to Estimate Inlier Points |
【本文地址】